RHIDP-13359: Lightspeed 1.10 changes#2210
Conversation
PR Build ResultsBuild passed -- 34/34 titles | 73s Content Quality AssessmentCQA Report
SummaryChecks: 19 total, 19 pass, 0 fail 19 checks: 19 pass, 0 fail Run Updated 2026-05-21 14:44:18 UTC |
pabel-rh
added
Peer review needed 📖
do-no-merge/review-in-progress 👀
Technical review needed 🔩
|
Once this PR is merged, we need to uncomment this line in this: |
| * Ollama (popular desktop inference server) | ||
| * vLLM (popular enterprise inference server) | ||
| * Gemini (available through Vertex AI) | ||
| The underlying {lcs-short} service integrates with several platforms that support the OpenAI API specification or utilize the vLLM inference engine. Because there is no explicit {rhoai-brand-name} provider option in the configuration, you must route those deployments through the vLLM or OpenAI-compatible provider settings. |
There was a problem hiding this comment.
@Jdubrick , I have rewritten this to apply your change. Would you please take a look?
There was a problem hiding this comment.
Yeah, I think a snippet in addition to that where it lets the users know that as long as it is OpenAI compatible in its API schema then they should be able to use the vllm provider type since it really just looks for OpenAI compatible and then looks for /v1 at the end of the url, but I can't make any guarantees, I'm just speaking from experience with other hosted providers
There was a problem hiding this comment.
Okay! Do we need to run this by anyone else to verify the new line I'm adding as per your comment?
The new line -
"The vllm provider type communicates with endpoints that conform to the OpenAI API schema by automatically appending /v1 to the configured provider URL. This mechanism allows you to use the vllm configuration for other hosted, OpenAI-compliant inference providers."
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="ollama-model-integration-for-local-development-environments_{context}"] | ||
| = Ollama model integration for local development environments |
There was a problem hiding this comment.
@Jdubrick , I've changed this based on your comment. Would you please take a look?
There was a problem hiding this comment.
I think I may have been confusing. You can use Ollama for cluster deployed environments if you want, you just need to make sure it is deployed in an environment that is accessible, if you just deployed it on localhost and tried to use it with a cluster deployed Lightspeed, it wouldn't work.
So:
Local Ollama + Local Lightspeed = good
Local Lightspeed + Cluster Ollama (if available externally from cluster) = good
Cluster Ollama + Cluster Lightspeed = good
Local Ollama + Cluster Lightspeed = bad
| @@ -0,0 +1,26 @@ | |||
| :_mod-docs-content-type: PROCEDURE | |||
There was a problem hiding this comment.
@gabemontero , would you please verify this section?
I also need help with the script for skopeo copy. I've left it empty now.
There was a problem hiding this comment.
you should work with the @redhat-developer/rhdh-install team on the skopeo copy script ... they can speak to it much more authoritatively on it than I can
There was a problem hiding this comment.
+1, consulting the @redhat-developer/rhdh-install is probably the best to make sure we don't miss anything related to air-gapping / how it works with Helm/Operator
There was a problem hiding this comment.
I've asked the RHDH-Install team to take a look.
gabemontero
left a comment
gabemontero
left a comment
There was a problem hiding this comment.
of course tweak my wording as needed @pabel-rh
but it is essential that we are clean on the additional need wrt updating the install pull secret for the kubelet
| * Ollama (popular desktop inference server) | ||
| * vLLM (popular enterprise inference server) | ||
| * Gemini (available through Vertex AI) | ||
| The underlying {lcs-short} service integrates with several platforms that support the OpenAI API specification or utilize the vLLM inference engine. Because there is no explicit {rhoai-brand-name} provider option in the configuration, you must route those deployments through the vLLM or OpenAI-compatible provider settings. |
There was a problem hiding this comment.
Yeah, I think a snippet in addition to that where it lets the users know that as long as it is OpenAI compatible in its API schema then they should be able to use the vllm provider type since it really just looks for OpenAI compatible and then looks for /v1 at the end of the url, but I can't make any guarantees, I'm just speaking from experience with other hosted providers
| {ls-short} supports the following inference provider configurations: | ||
|
|
||
| * OpenAI cloud-based inference services | ||
| * vLLM enterprise inference servers, which includes models hosted on {rhoai-brand-name} and {rhel} AI |
There was a problem hiding this comment.
Possibly here is where that explanation about it probably working via vllm for rhoai/rhelai
There was a problem hiding this comment.
I'm not familiar enough with those 2 services, I think maybe @gabemontero or @johnmcollier would know more about their workings?
| :_mod-docs-content-type: CONCEPT | ||
|
|
||
| [id="ollama-model-integration-for-local-development-environments_{context}"] | ||
| = Ollama model integration for local development environments |
There was a problem hiding this comment.
I think I may have been confusing. You can use Ollama for cluster deployed environments if you want, you just need to make sure it is deployed in an environment that is accessible, if you just deployed it on localhost and tried to use it with a cluster deployed Lightspeed, it wouldn't work.
So:
Local Ollama + Local Lightspeed = good
Local Lightspeed + Cluster Ollama (if available externally from cluster) = good
Cluster Ollama + Cluster Lightspeed = good
Local Ollama + Cluster Lightspeed = bad
| @@ -0,0 +1,26 @@ | |||
| :_mod-docs-content-type: PROCEDURE | |||
There was a problem hiding this comment.
+1, consulting the @redhat-developer/rhdh-install is probably the best to make sure we don't miss anything related to air-gapping / how it works with Helm/Operator
|
6 participants
IMPORTANT: Do Not Merge - To be merged by Docs Team Only
Version(s):
main, release-1.10
Issue:
RHIDP-13359
Preview:
Interacting with Red Hat Developer Lightspeed for Red Hat Developer Hub